According to Anthropic (@AnthropicAI), the company initially used a notoriously difficult take-home exam to assess prospective performance engineering candidates. This approach was successful in evaluating human applicants until their advanced AI model, Opus 4.5, managed to solve the exam, prompting a redesign of their assessment process. The blog post details how Anthropic is now focusing on creating AI-resistant technical evaluations, emphasizing the need for tests that both accurately measure human engineering skills and stay ahead of AI capabilities. This development highlights significant implications for AI-driven hiring processes and the broader challenge of designing assessments that distinguish between human and machine performance in technical roles. Source: Anthropic Engineering Blog (anthropic.com/engineering/AI-resistant-technical-evaluations)